An iPhone Weather App Interfaced via Bluetooth to an Arduino Sensor Board Running a TensorFlow Model

Alessio Tamburro


The source code for this article is available in this repository.

Note: the html version of this notebook was created with nbconvert and running the command jupyter nbconvert --to html --TagRemovePreprocessor.remove_input_tags="notvisible" notebook.ipynb. A tag "notvisible" was added to the cells that are not displayed in this rendered html

Arduino is an open-source electronics platform that provides both software and hardware to experiment with tiny machine learning, aka tinyML. An Arduino board can read inputs from sensors and turn them into output. In this project, we will use the on-board sensors for temperature, pressure and humidity. Pressure, humidity and temperature are sensors that are already integrated with Arduino. The objective is to use live readings and a model previously trained and deployed to the Arduino board to make predictions on the chance for precipitation. We will also build an iOS app and deploy it to an iPhone for acquiring the results from the board and displaying them. The app and the board do not need any Internet connection and therefore, the solution can be thought of as a companion to take to any remote region.

The code and notes in this notebook are only a summary of the work. The associated repository contains the full project.

Data Sourcing and Preparation

Sub-hourly data is publicly available from the NOAA. The documentation about the data is available here. We find the Iowa State University interactive portal an easy way to understand and browse through the weather data collected at NOAA. Data can be also downloaded directly from this site. However, sub-hourly data is not provided.

The NOAA data is originally in METAR format, typically used for reporting weather information. For a list of useful acronyms and codes this format uses, the reader can look at this document, for example.

The data needed for our purpose is the timestamp, pressure, hummidity, temperature and the precipitation.

Data Download

We download sub-hourly (5-min) data from the NOAA databases for 4 stations in the area of interest between 2014 and 2024. Station codes are available here (we use only ASOS stations).

The downloaded data is preprocessed. In particular, we created the function process_file to check the validity of the data and convert it from the METAR format (see function details in the repository). The humidity is calculated from the air temperature and dew point. We use backward and forward filling of missing values with the first available non-missing value at an earlier or later timestamp.

Station Data and Target Definition

The preprocessed data is saved in Parquet format. We will add the target to predict, which we define as 1 if we observe rain in the next 30 min (6 rows, each being 5 min) else 0.

Data Visualization

As we can see below, the dataset is highly imbalanced with only about 9% of the rows having precipitation values equal to 1.

We will use class weights to balance the dataset later during model training.

Typically, temperature, pressure and relative humidity are good predictors for precipitation although other factors that we do not consider here can improve the final results. In fact, our objective is beyond building an extremely accurate model. In any case, we can observe below how the occurrence of precipitation can be related to a decreasing pressure and temperature, and increasing humidity.

Training, Validation and Test Sets

Data Preprocessing

We preprocess the data before training the model. The input variables are standardized and we note down the means and standard deviations of the training dataset input variables for standardizing the new data coming from the Arduino sensors.

Data Generation for a TensorFlow Time Series Model

We reshape the data to create a time series forecasting model in TensorFlow.

We need to define a time window where we sequence the input data pressure, humidity and temperature that is collected every 5 min for the past 30 min. The target is the precipitation label in the next 30 min, which we labeled earlier PRECIPITATION_6 (6 5-min observations). In principle, we could also use the PRECIPITATION value for each of the 5-min observations in the past 30 min as an input but we need an additional sensor connected to the Arduino to also measure it.

Here below two examples of how our WindowGenerator labels the inputs and the output coming from the dataset.

The TensorFlow Model

We build a neural network model by searching for an optimal architecture that minimizes the Binary Focal Crossentropy loss function. This loss function seems to work well for this case with highly imbalanced labels. We also use the class weights defined earlier to mitigate the class imbalancement.

The Optimal Architecture

We use KerasTuner to search for the optimal architecture.

The Selected Architecture

Once we find the optimal neural network architecture, we train our model, we measure its performance and save it.

Model Performance

From the performance graphs below, it seems that our model underfits the data. However, we should keep in mind that we don't have many data exemplars with positive labels and during the splitting training/ validation/ test samples positive labels exemplars were randomly assigned to the splits. Some precipitation occurrences might have "unique" associated temperature, humidity, pressure values that are only see by one of the splits.

We can finally look at the ROC, which is not great.

The TFLite Model

TensorFlow Lite for microcontrollers is designed to run machine learning models on microcontrollers and other devices with only a few kilobytes of memory. A "hello world" example for Arduino is available here.

Typically, post-training integer 8-bit quantization is the main ingredient that make the trained model suitable for inference on devices with reduced memory computational capabilities such as an Arduino board. This widely adopted technique applies the quantization after training and converts the 32-bit floating-point weights into 8-bit integer values. However, we do not need to quantize the model we built due to its already small size. Quantizing the model will significantly reduce its predictive performance.

Test The TFLite Model

Since we did not quantize the model, the TFLite version should retain the original model's performance.

As expected, the performance remain the same after converting the TensorFlow model to a TFLite model.

Convert to C array for Arduino

The model binary data need to be converted to a constant C-byte array. The steps we need to follow are in the comments below. Since the model will be consumed by the Arduino processor, we will save the model directly into the folder that we set up later in the Arduino IDE for deployment.

Deploy to Arduino using the Arduino IDE

The deployment to the Arduino board is performed through the Arduino IDE that needs to be installed. We are not covering the development of the code needed to read the sensor data. The reader can refer to the repository for additional information and code (folder: arduino/iosWeather).

arduinoide

The Arduino board we use in this project is the Nano 33 BLE. Pressure, humidity and temperature are sensors already integrated with this Arduino. For information about the sensors, the user can look here.

ble33

Once we open the Arduino IDE, we create a new sketch (we call it iosWeather) and add the required libraries to it (Sketch > Include Librariy > Manage Libraries for a MacOS). In particular, we add the libraries to read from the sensors HS3003 (humidity, temperature) and LPS22HB (pressure). This can be done by searching the libraries in the Library Manager (see earlier snapshot).

We also add the model (model.h and model.cpp) to the sketch (Sketch > Add File).

Note: you need to install the TFLite libraries for Arduino (see the tutorials here and the examples). The TFlite libraries are manually installed into the Arduino library folder. The location for this folder varies by operating system, but typically it is in ~/Arduino/libraries on Linux, ~/Documents/Arduino/libraries/ on MacOS, and My Documents\Arduino\Libraries on Windows.

git clone https://github.com/tensorflow/tflite-micro-arduino-examples Arduino_TensorFlowLite

cd Arduino_TensorFlowLite

git pull

We can now use TFLite in our Arduino sketch. All the tensors the model consumes must be allocated, such as the input, output, and intermediate ones. The reason is that the TensorFlow Lite Interpreter does not allocate any memory at runtime.

The main .ino file must be written in C language.

ino1

and has two main functions

ino2

a setup function to initialize sensors, Bluetooth, arrays, tensors, and a loop function that streams the sensor data. We let the reader study the files in the repository to have more details about the structure.

With the Arduino board connected via USB to the computer, we can compile and upload the sketch. Note: uploading a new sketch or an empty sketch will replace the current sketch.

ADD IMAGE

Validate Arduino Results with Few Examples

Once the model and the Arduino script are deployed to the board, we can start testing them to validate the Arduino results match the results we obtain from running the model in our notebook.

For example, we take some rondom batches and shape the associated input and output arrays so that they match the arrays prepared in the Arduino script. Also, we run both the TensorFlow and TFLite models and save the predictions.

After we replace the actual sensor reading with the "simulated" readings we copy/ paste from here into the Arduino script, the predictions of the model deployed to the board should return the same results.

UUIDs for Arduino Bluetooth

Here we generate the Universally Unique Identifier (UUID) numbers that identify the service characteristics provided by the Bluetooth device. In the Arduino file, we can see the assignment of these UUIDs to the different characteristics we want to read and notify via the Bluetooth service.

BLEService weatherService("3c2312f7-7c24-4104-85ad-85a5a039d3cd");

BLEByteCharacteristic weatherCharacteristic( "4c40dfe1-7044-4058-8d53-24372c4e2e08", BLERead | BLENotify );

BLEFloatCharacteristic tempCharacteristic( "aaa691a2-3713-46ed-8d85-3a120d760175", BLERead | BLENotify );

BLEFloatCharacteristic humCharacteristic( "d85e531e-0d16-4e39-902c-417b100e867e", BLERead | BLENotify );

Similarly, the iosApp will have the same UUIDs used for the connectivity to the Arduino board via Bluetooth.

Notes on the iOS App

The app is developed using Xcode, a comprehensive suite of tools and resources that allow for designing, coding, and debugging iOS applications. When you open Xcode for the first time, it will ask what platform we want to develop for and download the required libraries.

xcodeprompt

We can now select “Create a new project” from the welcome screen and choose the app template from the iOS section. The reader can find the project in the repository (iosapp/ioasWeather). The code is in Swift language. SwiftUI is also available to help design the user interface.

xcodeinterface

Copyright (c) [2024] [Alessio Tamburro]

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

MIT license